
5 



Method and Apparatus for Defining and Correcting Image Data 



BACKGROUND OF THE INVENTION. 



1. Field of the Invention. 



10 



The invention, in general relates to a method and to an apparatus for 



defining and at least partially correcting image data and, more particularly, to the 
definition and correction of data used to form an image on a video or projection 
screen as well as on other electronic image forming devices. 

15 2. The Prior Art. 



It is often necessary and in some instances desirable to correct or 
otherwise affect the data used for forming an image on a video or projection 
screen of the kind used, for instance, in television sets, monitors, displays or 
20 projectors. The screen may be part of a cathode ray tube, or other plasma, 
liquid crystal display or micro reflector technology. Suitably adapted, the 
invention is also useful in connection with X-ray image forming technology, digital 
copiers, scanners, image transformers, overhead and other projectors and the 
like. 



The method is practiced to correct errors which typically occur in the 
reproduction of images. Such errors may be blurs, distortions, irregular 
illumination, lightness, brightness, hue, color and convergence. Practicing the 
methods leads do reduced demands on the mechanical, optical and electronic 
30 components such as cathode ray guns, LCD panels or projection lenses as well 
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as to a significant adjustment or calibration of such components. However, other 
shortcomings of projection and image forming screens as well as distortions 
resulting from oblique projections, known as Keystone distortions, may also be 
corrected. 

5 

The field of use of the invention is by no means restricted to the correction 
of image data errors, i.e. the reduction or elimination of undesirable changes in 
image data occurring in a real image reproduction systems as opposed to an 
ideal system. Basically, the invention may be used in connection with systems in 
10 which image data provided for image formation are subjected, prior to their input 
into the image reproduction system, to modifications definable by predetermining 
the result of modifying original image date into transformed image data, as, for 
instance, on the basis of test images. 

15 Hereinafter, "image reproduction system" is intended to refer to such 

systems in general. Moreover, "digital" and "digitized" image data or "digitized 
test image data" are not to be understood as image data existing primarily as 
digital image data, but also as image data prepared by digitizing primarily analog 
image data. 

20 

2. The Prior Art. 

A video projection apparatus is known from U.S. Patent No. 5.231,481, for 
instance, which is provided with a correction feature operating on the basis of 

25 feed-back for correcting errors in a projected image caused, for instance, by 
optical or electrical components of the projection apparatus. Feed-back is 
provided by a video camera which takes an exposure of a test image projected 
by the video projection apparatus. The exposure taken and stored by the video 
camera is compared to image data of the original of the test image which is 

30 deemed to be free of errors, for the purpose of calculating and storing corrective 
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signals. These corrective signals are utilized to control the control units of the 
projection apparatus in order to improve the quality of the projected image. 

A method disclosed by U.S. Patent No. 5,475,447 relates to the automatic 
5 correction of errors of convergence and distortion of an image projected onto a 
rectangularly framed projection surface by a video projector. The method is 
based of a video projector which is controlled by a video signal generator. A 
camera takes an exposure of the projection surface, and by means of a signal 
processor generates a video signal which is stored in an image storage. A 
10 computer calculates the coordinates of the four corner points of the rectangular 
projection surface and determines the position of reference points within the 
projection surface on the basis of the coordinates. At positions corresponding to 
i'3 the calculated positions of the reference points a signal generator then 

• J generates marked or tagged video signals. The video signal controls the video 

; |j 15 projector which projects an image including the marks. The digital camera stores 

the image including the marks in an image storage. A corrective signal for the 
automatic convergence and distortion correction of the camera is derived from 
any deviation between the positions of the reference points stored in the image 
storage and the positions of the stored marks. 



Etc: 
z 

m 20 



The video projection apparatus of U.S. Patent No. 5,091,773 is also 
provided with a correction feature operating on the basis of negative feed back 
for correcting, in a projected image, errors which may be caused, for instance, by 
the optical system. To this end, a test image consisting of light image points 

25 arranged as a raster is projected by the projected apparatus on a projection 
screen. An image of the image on the projection screen is formed on a 
substantially opaque filtering mask which is transparent only at those image 
points at which the image of an error-free test image displays light image points. 
Corrective signals for controlling the video projector are derived from the value of 

30 the light intensity which permeating the filtering mask. 
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A common feature of the known methods is that a corrective signal for 
improving the quality of the projected image is calculated on the basis of a test 
image stored by a digital camera for controlling the image generator of the 
projection system in order to correct errors of distortion or convergence. 

5 

The calculation of the corrective values is complex, and it is valid only for 
the analyzed image point sites. In respect of intermediate positions, the known 
methods require interpolations. This, too, is complex and provides no more than 
a mere approximation. Moreover, it is possible to correct only individual errors, 

10 primarily those relating to the geometry of an image. Other errors in the 

transmission characteristics of the reproduction system are not being addressed. 
The known methods require manipulation of control electronics or mechanical or 
optical system. In some instances it is even necessary to modify the image or 
projection screens. Another disadvantage is that the calculation of the corrective 

15 values has to be repeated each time an image is reproduction and for each 
image reproduction system used. 

This would seem to be of particular disadvantage where similar image 
reproduction systems known to be inflicted with substantially identical problems 
20 of image formation are used substantially simultaneously. 

OBJECTS OF THE INVENTION. 



It is, therefore, a general object of the invention to provide a method and 
25 related apparatus for forming images of superior quality. 

A more specific object relates to a method and to an apparatus which 
avoid the necessity of manipulating electronic, mechanical or optical systems of 
an image forming apparatus. 

30 
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Yet another object of the invention is to provide a method and an 
apparatus for forming images of superior quality by simply modifying image 
forming data to control all of the transmission properties of an image forming 
system. 

BRIEF SUMMARY OF THE INVENTION. 



In accordance with the invention, there is provided a method of defining 
and at least partially correcting errors of an image reproduction system by 
10 feeding digital image forming data to the input of a neuronal net which is 

implemented in appropriate circuitry for correcting errors and for forming input 
data for an image reproduction system, the parameters of the neuronal net 
having been defined by a preceding learning process which utilizes a test image 
of predetermined quality for generating target data. 

15 

In a preferred embodiment of the invention there is provided an apparatus 
including circuitry for accommodating a neuronal net the outputs of which are 
connected to the inputs of an image forming system and the parameters of which 
have been established in a preceding learning step based upon a test image of 
20 predetermined quality, a storage for image forming data connected to the inputs 
of the neuronal net, and an image recording device for generating digitized 
image data of an uncorrected image of the test image and connected to the 
inputs of the neuronal net for a time necessary to define the parameters of the 
neuronal net during the learning process. 

25 

Advantageously, the image recording device consists of a digital camera 
optionally connected to the neuronal net by way of an image data storage. 

The basic concept of the method in accordance with the invention is to 
30 utilize a neuronal net as a locally functioning digital filter for advance correction 
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with a mono-layered net being used where the stimulation characteristics of the 
image elements are linear. In terms of program structure the neuronal net may 
be implemented on a personal computer or in an application specific circuit. 

5 The target data, i.e. the learning pattern for the neuronal net is a digitized 

image of an uncorrected test image produced by the image reproduction system. 
The actual digital image data of the test image directly define the learning pattern 
for the training process. 

10 Following the learning process, the inputs of the neuronal net are fed with 

data of the image to be reproduced, and its outputs provide direct pixel input 
values for the image reproduction system which will thus be provided with 
corrected image data. In this manner, any errors in the transmission behavior of 
image formation, optics and projection of any images to be reproduced will have 

15 been checked and, if necessary, corrected so that as a result of having been 
compensated or corrected errors will no longer be visible on the projection or 
image forming screen. 

The special advantage of the invention is, therefore, that neither deflection 
20 nor intensity modulation of the image reproduction system require manipulation, 
or that the image or projection screens need by modified. 

The system is capable of simultaneously addressing several types of error 
at the same time. These may be image distortions and blurring caused by 

25 geometric distortions or optics which are out of focus, oblique projection 
(Keystone distortion) or curved projection screens. Also, irregular lightness 
distributions as may be caused by variations in the reflective properties of the 
projection screen or by the optical system or light source, will be corrected. In 
the case of colored image projection, errors in color which may occur at the 

30 transition between different colors (color seams) will also be corrected. A special 
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process maintains the color balance of the projector at surfaces of constant 
color. Defective alignment of color channels will be corrected as well (correction 
of divergence). 

Particular fields of application of the invention relate to display devices for 
direct and indirect projections by means of data and video projectors, to support 
screens of overhead projectors as well as to television and monitor display 
technologies. The invention may either be used to equip systems wherever they 
are being used or to calibrate apparatus during their final assembly. The 
invention may also be applied to stereoscopic image projections. 

DESCRIPTION OF THE SEVERAL DRAWINGS. 

The novel features which are considered to be characteristic of the 
invention are set forth with particularity in the appended claims. The invention 
itself, however, in respect of its structure, construction and lay-out as well as 
manufacturing techniques, together with other objects and advantages thereof, 
will be best understood from the following description of preferred embodiments 
when read in connection with the appended drawings, in which: 

Fig. 1 depicts a basic structure for practicing the invention; 

Fig. 2 is a schematic presentation of the training process in accordance 



Fig. 3 



with the invention; 
is a schematic presentation of a correction process in accordance 



Fig. 4a 



Fig. 4b 
Fig. 4c 



with the invention; 
depicts the specific structure of an artificial neuronal net for 
pre-correction; 

depicts the connection to an individual neuron; 



shows the realization of basic colors red, green and blue; 
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Fig. 5 is a typical test image for obtaining learning data. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS. 

5 The images to be reproduced (Fig. 1) are stored in digitized format in a 

computer connected to an image reproduction device such as a projector. The 
images may be rendered visible on an image or projection screen. It will be 
understood by those skilled in the art that the image reproduction device and the 
screen may be also be a cathode ray tube and a television screen, respectively.. 

10 

The method is carried out in two steps: The first step is a learning step 
(balancing), which is followed by the actual operation. An image recording 
device, e.g. an electronic camera, is used for balancing the correction and for 
taking an exposure of a test image which is stored in a computer or other 
15 appropriate circuitry. To avoid further errors, the image receiving device must be 
of a higher image than the image reproduction to be corrected. Otherwise, 
interfering image forming characteristics of the image forming device will have to 
be compensated prior to further processing. 

20 Fig. 4 is a simplified schematic presentation of an artificial neuronal net as 

a locally functioning digital filter of the kind used for the correction of image data. 
Assuming the overall system to be a linear one, the neuronal net used is a 
mono-layered one as shown in Fig. 4a. Fig. 4a is a simplified uni-dimensional 
rendition suitable for a single color channel. The inputs Xj constitute the pixel 

25 values of the image which are connected to each other by weights Wy as a result 
of the learning process. In an actual two-dimensional case, neurons are also 
connected to adjacent ones, but they are not confined to one line as shown in 
the example of a single neuron in Fig. 4b. For a comprehensive correction N 2 
neuronal nets of the kind shown in Fig. 4b are required for N color channels. Fig. 

30 4c depicts an example of a system of the primary colors red, green and blue 



Attorney Docket 000432 



(R,G,B). Each corrected color channels R\ G\ B' thus results from the sum of 
the outputs of the neuronal nets RR,RG,RB,BR,BB,BG,GR,GG,GB fed by 
channels R, G and B. 

5 Balancing of the system for correction is carried out by training the 

neuronal net with a computer. The test image which contains the errors to be 
corrected is digitally stored as a teaching pattern and is used to gather learning 
data for the artificial neuronal network. Since the test image is stored free of 
errors in the computer, it directly defines the training pattern of the training 
10 process. 

The test image may be a relatively simple structure (Fig. 5). The artificial 
neuronal net derives the parameters required for the correction from the target 
data in connection with the learning pattern, by defining the weights w y of the 
15 neuronal net on the basis of comparing the outputs of the neuronal net y with the 
learning pattern. In a manner distinct from U.S. Patent 5,091 ,773 the training 
process provides a localized sliding detection of corrective values, thus avoiding 
the need for interpolation. 

20 Once the learning step is completed, the inputs of the neuronal net will be 

fed data of an image to be reproduced, and the outputs of the neuronal net will 
be pixel values for direct input to an image reproduction device which is thus 
stimulated or energized by corrected image forming data. The conversion 
program of the balanced correction system may be provided by a conventional 

25 PC or by an application specific circuit. In a manner distinct from European 
patent application 934,653, no manipulation of the deflection and intensity 
modulation of the image forming device is necessary. 

During the ongoing operation (Fig. 3) corrected images are produced such 
30 that no more errors are visible on the projection or image screen. The correction 
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